A Comparative Evaluation of Deep Learning Architectures for Binary Network Intrusion Detection Using the NSL-KDD Dataset

Abstract

The rapid growth of digital communication technologies, cloud computing, and Internet of Things (IoT) devices has increased both the frequency and sophistication of cyber-attacks, making effective intrusion detection an essential component of modern cybersecurity systems. Traditional signature-based intrusion detection systems (IDS) are effective against known attacks but fail to detect previously unseen or evolving threats. This study investigates the application of deep learning models for binary network intrusion detection using the NSL-KDD benchmark dataset. Three standalone architectures, Convolutional Neural Networks (CNN), Long Short-Term Memory (LSTM) networks, and Deep Neural Networks (DNN), are implemented and evaluated, alongside a CNN-LSTM Hybrid model that integrates spatial and sequential learning, and a DNN-LSTM Ensemble model that combines independently trained DNN and LSTM predictions through weighted averaging. Following data cleaning, categorical encoding, normalization, and Random Forest-based feature selection (41 features reduced to 20), all models were trained and evaluated under identical conditions using Accuracy, Precision, Recall, F1-Score, ROC-AUC, training time, and inference time. The standalone DNN model achieved the best overall performance, with 80.98% accuracy, 97.08% precision, 68.66% recall, 80.43% F1-score, and 96.11% ROC-AUC, while also requiring the shortest training time (39.69 s). The CNN-LSTM Hybrid model attained the highest precision (97.23%) but did not outperform the standalone architectures overall, and the DNN-LSTM Ensemble produced balanced but not superior results. These findings indicate that carefully designed standalone architectures can match or exceed the performance of more complex hybrid and ensemble models for binary intrusion detection, while incurring substantially lower computational cost. The study contributes a controlled, common-framework comparison of five deep learning architectures and provides practical guidance for selecting computationally efficient models for anomaly-based intrusion detection.

Introduction

The rapid growth of cloud computing, Internet of Things (IoT) devices, mobile applications, online banking, and smart infrastructure has significantly increased the complexity of network traffic and expanded the opportunities for cyberattacks. Modern cyber threats—including Distributed Denial-of-Service (DDoS) attacks, malware, ransomware, phishing, botnets, insider attacks, and Advanced Persistent Threats (APTs)—pose serious risks to the confidentiality, integrity, and availability of information systems. Consequently, effective Intrusion Detection Systems (IDS) are essential for protecting computer networks against increasingly sophisticated attacks.

Traditional IDS techniques are classified as either signature-based or anomaly-based systems. Signature-based IDSs detect attacks by matching network traffic with predefined attack signatures, providing high accuracy for known threats but failing to identify new or zero-day attacks. Anomaly-based IDSs overcome this limitation by learning patterns of normal network behavior and detecting deviations. Recent advances in machine learning and deep learning have greatly improved anomaly detection by enabling automatic extraction of complex patterns from network traffic without relying on manually engineered features.

Previous studies have successfully applied deep learning architectures such as Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), Long Short-Term Memory (LSTM) networks, Deep Neural Networks (DNNs), and Autoencoders for intrusion detection. Hybrid architectures combining CNNs and LSTMs, attention mechanisms, Transformer models, and Graph Neural Networks have also demonstrated promising results. However, most existing research evaluates only a single architecture under different preprocessing and feature selection methods, making direct comparison difficult. Furthermore, it remains unclear whether hybrid or ensemble models consistently outperform well-designed standalone architectures. To address this research gap, the study performs a controlled comparison of five deep learning models—CNN, LSTM, DNN, CNN-LSTM Hybrid, and DNN-LSTM Ensemble—using identical preprocessing, feature selection, and evaluation procedures on the NSL-KDD benchmark dataset.

The proposed framework consists of five stages: dataset acquisition, data preprocessing, feature selection, model development, and comparative performance evaluation. The NSL-KDD dataset, an improved version of the KDD Cup 1999 dataset with reduced redundancy and better class balance, serves as the benchmark dataset. It contains 125,973 training records and 22,544 testing records, each described by 41 network traffic features and a class label. The original multiple attack categories—Denial of Service (DoS), Probe, Remote-to-Local (R2L), and User-to-Root (U2R)—are merged into a single Attack class, creating a binary classification problem distinguishing Normal (0) and Attack (1) traffic.

Data preprocessing includes converting the multiclass labels into binary classes, encoding categorical features such as protocol type, service, and connection flag using label encoding, normalizing numerical attributes to the range of 0–1 using Min-Max normalization, and partitioning the dataset into training, validation, and testing subsets. To reduce computational complexity while preserving discriminative information, feature selection is performed using Random Forest feature importance and correlation analysis. The twenty most important features, including src_bytes, dst_bytes, same_srv_rate, dst_host_same_srv_rate, flag, protocol_type, and other host-related statistics, are retained, reducing the original feature space by 51.22%.

Five deep learning models are developed using the same twenty selected features, Adam optimizer, binary cross-entropy loss function, and consistent training procedures. The CNN employs two one-dimensional convolutional layers followed by batch normalization, max pooling, dropout, and dense layers to learn spatial feature relationships. The LSTM consists of two stacked LSTM layers capable of modeling temporal dependencies in network traffic. The DNN is a fully connected neural network containing three hidden dense layers with batch normalization and dropout, representing the simplest architecture among the five models. The CNN-LSTM Hybrid combines convolutional layers for spatial feature extraction with an LSTM layer for sequential learning within a single architecture. The DNN-LSTM Ensemble independently trains the DNN and LSTM models and combines their prediction probabilities using weighted averaging, with the optimal weight configuration (0.7 DNN, 0.3 LSTM) selected based on validation performance.

The experiments are implemented using Python, TensorFlow/Keras, Scikit-learn, Pandas, and NumPy within a Google Colaboratory GPU environment. All models are trained and tested using identical datasets and feature sets to ensure that performance differences are solely due to architectural characteristics. Model performance is evaluated using accuracy, precision, recall, F1-score, ROC-AUC, as well as training time and inference time to assess computational efficiency.

Experimental results demonstrate that all five models achieve high precision, exceeding 95%, indicating a low false-positive rate. Among the individual models, the CNN achieves 77.10% accuracy, 96.01% precision, and 91.06% ROC-AUC, but records the lowest recall (62.36%), suggesting that spatial feature extraction alone misses many attacks. The LSTM improves recall to 68.67%, resulting in 80.43% accuracy and an F1-score of 79.98%, confirming the importance of temporal information in intrusion detection.

The Deep Neural Network (DNN) delivers the best overall performance, achieving the highest accuracy (80.98%), precision (97.08%), F1-score (80.43%), and ROC-AUC (96.11%), while also requiring the shortest training time (39.69 seconds). These results indicate that a well-designed fully connected network can effectively model nonlinear relationships within the selected network features without requiring the additional complexity of convolutional or recurrent architectures.

The CNN-LSTM Hybrid achieves the highest precision (97.23%) and a strong ROC-AUC (95.47%), demonstrating its effectiveness in minimizing false positives. However, its overall accuracy (78.36%) and F1-score (77.04%) remain below those of the standalone DNN and LSTM models, while its training time (120.16 seconds) is the highest among all architectures. Similarly, the DNN-LSTM Ensemble achieves competitive performance (80.31% accuracy, 79.62% F1-score, 94.03% ROC-AUC) but fails to outperform the standalone DNN and requires the highest inference time (5.24 seconds) because both constituent models must be evaluated during prediction.

Comparative analysis reveals that recall remains the primary limitation for all models, with even the highest-performing LSTM detecting only 68.67% of attack instances at the default classification threshold. Nevertheless, ROC-AUC values above 0.90 for all models indicate strong discrimination capability that could potentially be improved through threshold optimization. High precision across all architectures suggests a low false alarm rate, reducing alert fatigue for security analysts in operational environments.

The most significant finding of the study is that increasing architectural complexity does not necessarily improve intrusion detection performance. Neither the CNN-LSTM Hybrid nor the DNN-LSTM Ensemble consistently outperformed the simpler standalone DNN despite requiring substantially greater computational resources. This demonstrates that, when effective preprocessing and feature selection are employed, a relatively simple Deep Neural Network can provide the best balance between detection accuracy, computational efficiency, and practical deployment. Overall, the study offers a comprehensive comparison of five deep learning architectures under identical experimental conditions and provides valuable guidance for selecting efficient and effective deep learning models for network intrusion detection systems.

Conclusion

This study presented a controlled comparative evaluation of five deep learning architectures — CNN, LSTM, DNN, a CNN-LSTM Hybrid, and a DNN-LSTM Ensemble — for binary network intrusion detection on the NSL-KDD dataset, using identical preprocessing, a common twenty-feature subset selected via Random Forest importance analysis, and consistent evaluation metrics. The standalone DNN achieved the best overall performance (80.98% accuracy, 97.08% precision, 68.66% recall, 80.43% F1-score, 96.11% ROC-AUC) with the lowest training time among all models, while the CNN-LSTM Hybrid achieved the highest precision and the LSTM achieved the highest recall. Notably, the hybrid and ensemble models did not outperform the standalone DNN despite their added architectural complexity and computational cost, indicating that model complexity is not, by itself, a reliable predictor of intrusion detection performance. These findings support the practical value of well-designed standalone architectures for binary intrusion detection when computational efficiency is a priority, and provide a controlled reference point for future comparative work. Limitations of this study include reliance on a single benchmark dataset (NSL-KDD), a binary classification scope, a fixed twenty-feature subset, and limited hyperparameter search. Future work will extend this framework to multi-class attack classification (DoS, Probe, R2L, U2R), cross-dataset validation using UNSW-NB15, CICIDS2017, and Bot-IoT, attention-based and Transformer architectures, adaptive/dynamic ensemble weighting, explainable AI techniques (e.g., SHAP, LIME) for interpretability, and evaluation in real-time deployment settings.

References

[1] Khairi ASMA, Nugroho EP, Rachman JR. Implementation of signature based intrusion detection system with Snort rule on E-voting system. Journal of Computers for Society. 2023;4(1):17-36. https://doi.org/10.17509/jcs.v4i1.71176 [2] Díaz-Verdejo J, Muñoz-Calle J, Estepa Alonso A, Estepa Alonso R, Madinabeitia G. On the detection capabilities of signature-based intrusion detection systems in the context of web attacks. Applied Sciences. 2022;12(2):852. https://doi.org/10.3390/app12020852 [3] M. Tavallaee, E. Bagheri, W. Lu, and A. A. Ghorbani, \"A detailed analysis of the KDD CUP 99 dataset,\" Proc. IEEE Symp. Computational Intelligence for Security and Defense Applications, pp. 1–6, 2009. [4] Y. Xin, L. Kong, Z. Liu, Y. Chen, Y. Li, H. Zhu, M. Gao, H. Hou, and C. Wang, \"Machine Learning and Deep Learning Methods for Cybersecurity,\" IEEE Access, vol. 6, pp. 35365–35381, 2018. [5] W. Wang, M. Zhu, J. Wang, X. Zeng, and Z. Yang, \"End-to-End Encrypted Traffic Classification with One-Dimensional Convolution Neural Networks,\" IEEE Int. Conf. Intelligence and Security Informatics, pp. 43–48, 2017. [6] Vinayakumar R, Alazab M, Soman KP, Poornachandran P, Al-Nemrat A, Venkatraman S. Deep learning approach for intelligent intrusion detection system. IEEE Access. 2019;7:41525-41550. https://doi.org/10.1109/ACCESS.2019.2895334 [7] J. Kim, J. Kim, H. L. T. Thu, and H. Kim, \"Long Short Term Memory Recurrent Neural Network Classifier for Intrusion Detection,\" Int. Conf. Platform Technology and Service, pp. 1–5, 2016. [8] M. Alauthman, N. Aslam, M. Al-Kasassbeh, S. Khan, A. Al-Qerem, and K. Choo, \"An Efficient Reinforcement Learning-Based Botnet Detection Approach,\" Journal of Network and Computer Applications, vol. 150, 2020. [9] N. Shone, T. N. Ngoc, V. D. Phai, and Q. Shi, \"A Deep Learning Approach to Network Intrusion Detection,\" IEEE Transactions on Emerging Topics in Computational Intelligence, vol. 2, no. 1, pp. 41–50, 2018. [10] J. An and S. Cho, \"Variational Autoencoder Based Anomaly Detection Using Reconstruction Probability,\" Special Lecture on IE, vol. 2, no. 1, pp. 1–18, 2015. [11] H. Hindy, D. Brosset, E. Bayne, A. Seeam, C. Tachtatzis, R. Atkinson, and X. Bellekens, \"A Taxonomy and Survey of Intrusion Detection System Design Techniques, Network Threats and Datasets,\" IEEE Communications Surveys & Tutorials, vol. 24, no. 2, pp. 1105–1141, 2022. [12] A. Ferrag, L. Maglaras, H. Janicke, J. Jiang, and T. Shu, \"Deep Learning for Cyber Security Intrusion Detection: Approaches, Datasets, and Comparative Study,\" Journal of Information Security and Applications, vol. 50, 2020. [13] Y. Li, R. Ma, and R. Jiao, \"A Hybrid Malicious Code Detection Method Based on Deep Learning,\" International Journal of Security and Its Applications, vol. 9, no. 5, pp. 205–216, 2015. [14] Ullah S, Boulila W, Koubaa A, Ahmad J. Attention-based hybrid deep learning model for intrusion detection in IIoT networks. Procedia Computer Science. 2024;246:3323-3332. https://doi.org/10.1016/j.procs.2024.09.307 [15] N. Shone, T. N. Ngoc, V. D. Phai, and Q. Shi, \"Deep Learning Approach Combining Autoencoders and Deep Belief Networks for Intrusion Detection,\" IEEE Transactions on Emerging Topics in Computational Intelligence, vol. 2, no. 1, pp. 41–50, 2018. [16] A. Ferrag, L. Maglaras, S. Moschoyiannis, and H. Janicke, \"Deep Learning for Cyber Security Intrusion Detection: A Review of Transformer-Based Models,\" Journal of Information Security and Applications, vol. 68, 2022. [17] Z. Wu, S. Pan, F. Chen, G. Long, C. Zhang, and S. Yu, \"A Comprehensive Survey on Graph Neural Networks,\" IEEE Transactions on Neural Networks and Learning Systems, vol. 32, no. 1, pp. 4–24, 2021.

Copyright

Copyright © 2026 Ketki Naik, Sanjeev Ghosh. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.